Free-hand sketch-based image retrieval (SBIR) is a specific cross-viewretrieval task, in which queries are abstract and ambiguous sketches while theretrieval database is formed with natural images. Work in this area mainlyfocuses on extracting representative and shared features for sketches andnatural images. However, these can neither cope well with the geometricdistortion between sketches and images nor be feasible for large-scale SBIR dueto the heavy continuous-valued distance computation. In this paper, we speed upSBIR by introducing a novel binary coding method, named \textbf{Deep SketchHashing} (DSH), where a semi-heterogeneous deep architecture is proposed andincorporated into an end-to-end binary coding framework. Specifically, threeconvolutional neural networks are utilized to encode free-hand sketches,natural images and, especially, the auxiliary sketch-tokens which are adoptedas bridges to mitigate the sketch-image geometric distortion. The learned DSHcodes can effectively capture the cross-view similarities as well as theintrinsic semantic correlations between different categories. To the best ofour knowledge, DSH is the first hashing work specifically designed forcategory-level SBIR with an end-to-end deep architecture. The proposed DSH iscomprehensively evaluated on two large-scale datasets of TU-Berlin Extensionand Sketchy, and the experiments consistently show DSH's superior SBIRaccuracies over several state-of-the-art methods, while achieving significantlyreduced retrieval time and memory footprint.
展开▼
